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ABSTRACT 

The aim of the studies reported in this paper is to gain classroom based empirical evidence 
on the learning effectiveness of learning objects used in two types of study settings: 
Collaborative and individual. A total of 127 seventh and ninth grade students participated in 
the experiments. They were assigned into one of the study modes and worked with one of the 
four learning objects. The pretest and post test measures and observation data showed that 
though the learning objects were primarily developed for individual use, they may also help 
students working in peers. The comparison among different study modes did not strongly 
favor any of the study modes. However, the study provided evidence that using the learning 
objects in different modes is possible and designing and developing learning objects for the 
use of multiple modes is crucial. Implications on collaborative learning objects are discussed. 
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INTRODUCTION 

Many technology based materials only distribute content presentation with highly colorful, 
maybe animated, graphical representations. In widely available courseware sets, learning 
facilities for exploration, and virtual manipulatives for provoking thinking and reflection are still 
limited. With the increased use of Internet tools, chat, forums and discussion boards have 
become more common than those sense making and student centered elaborative tools. 
Also, the use of digital technology based communication tools unfortunately have become 
sociability tools rather than social learning tools. In recent years, technological advancements 
together with instructional design have contributed to the development of new educational 
materials to improve student learning. Though with the introduction and industrial standards of 
learning objects (LOs), technology based learning materials have become more and more 
individualistic, ease of access and ease of use of communication tools have widened the gate 
for collaborative learning materials. The streams of LOs and collaborative learning tools seem 
to have travelled different routes. However, as learning is an individual process, no matter in 
what educational setting it takes place, whether collaborative or self-study, all available 
learning materials including LOs should be employed in any educational setting. 

Theoretical analyses of classroom group work may be traced back to Dewey's assertion 
(1916) that students should be stimulated to work as participants of communities, actively 
pursuing interests in cooperation with others. Later Piaget (1932) assumed that cooperation 
provides the social context where pupils would be motivated to organize existing thoughts 
with alternatives. Vygotsky (1978) further stressed the prominence of learning within a social 
context. His argument on the social dimension in individuals' construction of knowledge and 
meaning led to the notion that collaboration with peers assists learners reaching new 
knowledge. In contrast to individual learning, collaborative learning was defined as (Pfister, 
2005, p.40) “a learning method involving a group of learners who exchange knowledge and/or 
solve problems together and interdependently under a common learning goal”. In such 
learning process, knowledge will be elaborated when learners need to (1) make their 
knowledge, misconceptions, and lack of knowledge explicit (King, 1999; Pfister, 2005), and 
(2) negotiate knowledge through arguments and justifications with other participants during a 
discourse (Anderson, Bruhn, Grasel, & Mandl, 2001; Driver, Newton, & Osborne, 2000). More 
elaborated knowledge via collaborative activities can lead to shared understanding, deep 
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learning, critical thinking, and long term retention of the learned material (Fisher Bruhn, 
Grasel, & Mandl, 2002; Johnson, & Johnson, 1994; Porcaro, 2011). 


COLLABORATIVE LEARNING RESEARCH and LOs 

The studies on collaborative learning have lead to controversial results with respect to its 
effectiveness. Though a number of studies on collaborative learning have revealed promising 
improvement in learning outcomes for both group and individual basis, the empirical research 
has also shown disadvantages of collaborative settings. For example, evidence was provided 
on second grade students' use of collaborative learning, particularly in that the technological 
network improves communication, interactivity, negotiation and coordination between 
members of collaborative learning groups (Infante, Hidalgo, Nussbaum, Alarcon, & Gottlieb, 
2009). In another study where secondary school students used wikis for a collaborative work; 
the study reported learning gains and concluded that rather than the medium, “consideration 
for the authenticity of the task, the role of the teacher, and the nature and form of assessment 
will be essential if a move to more collaborative practices in the classroom is desired” (p. 
106). 

There are studies (Anderson et al., 2001; Pfister, 2005) providing evidence in terms of 
drawbacks of collaborative learning, for example, it was reported that (Ding, 2009), in 
computer supported collaborative learning (CSCL), one student was sometimes put at a 
disadvantage while the group succeeded. Even if a consensus was reached, there was 
always the possibility that one student might not have cognitive involvement, hence poor 
learning may result. It was also reported (Edwards, Coddington, & Caterina, 2009) that “some 
students work in a more consensual fashion than the others, debating their plans together 
before acting. Some tended to act without discussing their ideas with their partners, and often 
took hierarchical, complementary roles rather than egalitarian, cooperative ones” (p. 46). The 
results of the research studies in CSCL are hard to compare, because they differ from each 
other in terms of properties. A detailed analysis of research on different collaborative learning 
environments is beyond the scope of this paper. 

Bringing learners together or placing them in groups does not guarantee collaboration (Fisher 
et al., 2002, Tolmie, Topping, Christie, Donaldson, Howe, Jessiman, Livingston & Thurston, 
2009); however “a complex of simultaneously applied instructional approaches, each 
reinforcing and complementing the other can enhance collaborative learning and social 
interaction among group members” (Kirschner & Kreijns, 2005; p. 172). Many researchers 
relate the notion, design and implementation of LOs to e-learning (Varlamis & Apostolakis, 
2006; Cohen & Nycz, 2006) where students use the provided learning materials mostly on 
their own and seldom collaboratively (Ada, 2009; Sabau, 2007). Some of the standardization 
efforts have made LOs an approach to learning and teaching. The LO approach became 
popular due to its features such as interoperability, durability, scalability, adaptability, 
reusability, flexibility and cost-effectiveness. For that approach, the sharable content object 
reference model (SCORM), aims to provide access to materials tailored to individual needs, 
delivered cost-effectively anytime and anywhere. All elements in SCORM are accompanied 
by meta-data defined according to a standard called learning object metadata. These 
standards and SCORM framework neglects the social interaction principle of knowledge 
integration (Akpinar & §imsek, 2007). Further, there is no group in SCORM, identifying 
relations between users is not considered, and therefore aggregating individual users’ 
interaction data into a group's interaction data is not possible. Maybe due to weakness in 
industrial standards of LOs; design, development and evaluation of collaborative use of LOs 
have not sufficiently been studied. 
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PURPOSE OF THE STUDY 

A few studies have worked on the design, development and evaluation of collaborative LOs. 
For example; Nurmi and Jaakkola investigated the collaborative use of LOs with only one 
group where students studied in pairs, and reported that students who collaboratively studied 
the content with LOs benefitted more from the LOs (Nurmi & Jaakkola, 2006). Many LO 
repositories contain LOs developed for individual use rather than collaborative use. However, 
due mainly to the popularity of collaborative learning, many teachers use those LOs, 
developed for individual learning, as collaborative learning materials. 


This study seeks to reveal classroom based evidence that LOs developed for individual use 
will not be helpful when used in a mode different from their actual design purpose. Hence, the 
learning effectiveness of LOs, developed for individual use, used in two types of study 
settings (collaborative and individual) was are investigated. Effectiveness of the LO based 
learning environments in the two settings are then compared. 


METHODOLOGY 

This mixed method research incorporated a quasi-experimental design with pre and post 
tests. The study collected and analyzed quantitative and qualitative data. The sample of the 
study consisted of 102 7 th graders and 25 9 th graders, studying in four different state schools 
located in suburban areas of a metropolitan city. It is important to note that state schools in 
Turkey differ from private schools in many ways: As the Turkish education system is 
centralized, and the Ministry of Education decides funding, curriculum, educational policies, 
textbooks, teacher appointments, and course schedules; classroom activities in all grades are 
strongly influenced by the curriculum guides of the Ministry. Whereas in private schools, the 
Ministry has control over governance of the school activities, the schools are free to extend 
the curriculum, to follow different policies and to adjust its curriculum scheduling. In terms of 
ICT usage in classrooms, private schools, due to their rich facilities, integrate technology into 
their activities more often than state schools. There is substantial evidence that Turkish 
teachers in state schools apply chiefly traditional methods of teaching and learning (Altun, 
2006; Kurt, 2010). 

One group of 7 th graders studied a module on unit of Pressure/Buoyant force came from two 
classrooms, taught by the same teacher, at the same school. The two other learning units for 
the 7th graders, taught by different teachers, were studied in one classroom each from two 
different schools. Another set of 9 th graders were also from a classroom of a different school. 
In selecting the sample, it was ensured that the students had not yet studied the content of 
the LO at that grade. The sample was selected in collaboration with classroom teachers who 
allowed access to their class activities, and had not covered content of the LO at the time the 
sample selection procedure started. Therefore, a practical sampling was preferred due to 
accessibility. The students' allocation to the groups of individual and collaborative study 
modes were carried out with simple random selection: The LO on Pressure/Buoyant force 
was to be studied by two classrooms using the same computer laboratory, the first classroom 
became the individually studying group and the second became the collaboratively studying 
group. The other LOs were studied by one classroom each, for that reason the number of 
computer stations at the computer labs helped to allocate students to the study modes, 
placing first entering students in the collaborative study modes. Table 1 summarizes some 
properties of the sample, and the LOs studied. 

Materials 


To compare effectiveness of individual and collaborative usage of LOs, four LOs for four 
learning units, difficult to learn, were selected from a LO repository. Three of the LOs were on 
seventh grade Science, Pressure/Buoyant force; Work and energy, and Electricity in daily life, 
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and one was on ninth grade Geography, Map scales. The LOs with simulation characteristics 
were selected. The tasks in the LOs required students to study a given problem within the 
simulated story of the LO, and answer the given contextualized problem by using 
computational facilities of the particular LO. In addition, two achievement tests for each of the 
learning units were developed; one as a pre-test to measure how much students know about 
the learning content, another as a post-test to measure how much students learnt about the 
content. The two tests for the units targeted the same instructional objectives, had the same 
number of multiple choice items, however the items were different. Number of items in the 
tests is given in Table 1. 

Procedure 

The experimental studies took place in the computer labs of the schools. The purpose of the 
studies was explained as “helping the students to learn the given learning task by using the 
presented LO settings” to all participants. Each group was then given the pretest lasting 10-20 
minutes for four different learning units. Following the pretest, the participants were asked to 
run the courseware. The procedures to control and use the facilities of the courseware 
settings were explained to them which took 4-8 minutes. The students who were going to 
study in a collaborative mode, the two students working together in front of a computer, were 
instructed to “do the activities in the courseware setting together: Study the materials, then 
share with your partner what you understood and what your opinion is, discuss with the 
partner and finally produce an agreement. All decisions you have to make in using the system 
and in your study in the system must be agreed on, even all mouse actions and any data 
entry to the system must be based upon agreement with the partner”. Further, in order to 
make the engagement time similar for the two learning groups, the students who were going 
to study in an individual mode were instructed to “explain yourself what you did understand 
and why you made the particular decision at a given task of the LO”. 


Table 1: Some properties of the sample, and the LO studied 


Learning 

object 

domain 

Learning object on 

Grade 

Age 

Number 

of 

activity 

Size of 
individually 
studying 
group 

Size of 

collaboratively 
studying group 

Number of 
classroom 

Science 

Pressure Buoyant 
force 

7 

13 

10 

25 

2S 

2 

Science 

Work and energy 

7 

13 

14 

7 

14 

i 

Science 

Electricity in daily 
life 

7 

13 

16 

12 

16 

i 

Geography 

Map scales 

9 

15 

10 

15 

10 

i 


During the students' activities, the researcher took notes on students' performances regarding 
usage of the system. The notes included information about how the students handled the 
screen objects, and how they navigated the facilities. The dynamics of pair interaction 
regarding learning was conversational in nature, and could not be addressed in the notes. 
The researcher's present in the classroom continuously reminded students that both 
members of a pair is responsible for task solutions and they have to agree on all their actions. 
Flence a consensus agreement would be built. Throughout the activities, the researcher 
walked around the classroom and closely monitored the pairs' activities to make sure that no 
one was dominating the peer interaction; those who didn't consider their peer's suggestions 
were warned to do so. The researcher managed to monitor these activities due to the small 
number of pairs (at most eight pairs) in one lab session. To allow students to try out more 
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ideas than the agreed ones in the LO settings, they were asked to do the following trial of an 
agreed idea for the task. Because the essence of collaboration among the paired peers is the 
construction of shared meanings for conversations, concepts, and experiences, thinking 
should be distributed among members of the pairs. In collaborative learning, learner 
interaction includes shared as well as individual responsibility, and identification of their role in 
the learning task (Ewing & Miller, 2002). For that reason, in the studies, each member of a 
pair first agreed on who would control the manipulation devices, mouse and keyboard (in fact 
in some pair groups, one of the students controlled the mouse while the other controlled the 
keyboard), then they read the presented task and thought about how to do it in the LO. They 
were asked to share with each other what action they would take, and what particular tools of 
the LO they would use. In this way, each member of the pair worked on the same aspect of 
the problem at the same time, sharing cognitive responsibility for the task at hand. 

After the learning activities with the LOs ended, each group then received their post-test 
lasting 8-20 minutes for four different learning units. In the pre and post tests, all students 
answered the items on their own. In addition, a small usability questionnaire for each LO was 
given to the students after the study. The questionnaires specific to each LO were developed 
in the form of a rating scale considered five different topics: system screen, terms and system 
information, learning to operate the system and system capabilities, and overall user reaction. 
The scales use a Likert type rating scheme (three-point) based on the suitability of the tool for 
performing various tasks. The scales consisted of 15-20 items, its key is as follows: Disagree, 
Neutral, Agree. 


Table 2: Descriptive statistics on the pre and post test scores of the groups 


Learning 

object 

Study 

mode 

n 

Maximum 
score to be 
attained 

Pretest 

Mean Stdev. 

Posttest 

Mean Stdev. 

Posttest-Pretest 
Difference 
Mean Stdev. 

Cohen’s d effect 
size for 
Posttest-Pretest 

1. Pressure, B 

n 

28 

10 

4,00 

1,96 

3,90 

1,70 

-1,03 

2,00 

0,24 

uoyant force 

i 

25 

3,82 

1,91 

3,89 

2,10 

0,07 

2 32 

0,05 

2. Work and 

n 

14 


3,64 

1,39 

5,07 

2,95 

1,42 

2,71 

0,60 

Energy 

i 

7 

10 

4,29 

2,21 

6,57 

1,13 

2 : 2S 

2,29 

1,21 

3. Electricity 

n 

16 


8,00 

1,97 

7,19 

3,27 

-0,81 

2,56 

2,56 

in daily life 

i 

12 

16 

7,50 

1,93 

9,25 

2,67 

1,75 

2,41 

0,72 

4. Map scales 

n 

10 

10 

2,60 

1,26 

3,40 

0,97 

0,80 

1,75 

0,07 

i 

15 

2,00 

1,20 

2,60 

1,30 

0,60 

1,96 

1,87 


I. Individual: 11. Collaborative 


FINDINGS AND DISCUSSIONS 

To examine whether students' two modes of activities within the LOs helped them to learn the 
contents, a series of statistical analyses were conducted on the pre and post scores. The 
distribution of scores for each group in the four studies did not show normality. Accordingly, a 
non-parametric, Mann-Whitney U, test was conducted to verify differences. Before the 
statistical tests, the data were descriptively examined: In all four applications, the individually 
and collaboratively studied students' pre and post test scores were similar (see Table 2). 
Conversely, there are inconsistencies in the students’ progress indicated by the pre and post 
test scores. The individual studied students showed more progress than collaboratively 
studied students in the LOs for Pressure/Buoyant force and Electricity in daily life, though the 
difference is not significant. Also, while the effect size for the groups studying LO for 
Pressure/Buoyant force is negligible, it is remarkable in the LO for Electricity in daily life. In 
fact, the collaboratively studied groups scored lower means in the post test than in their pre 
test. Further, while the collaborative studied group in the Geography LO demonstrated a 
higher mean score in the post test than the pre test, the effect size for the group is negligible. 
According to Cohen’s d effect size measures, in two of the studies, LO for Work and Energy, 
and LO for Map scales, the instructional effects of the two LOs were exceedingly higher for 
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the individually studied students than collaboratively studied ones. In only one study, LO for 
Electricity in daily life, the instructional effect of the LO were higher for the collaboratively 
studied students than individually studied ones, but nevertheless that effect was negative. 

The comparison between the individual and collaborative studied groups' pre and post test 
scores demonstrated that the groups' pre and post test differences did not significantly differ 
in three of the studies, with the seventh grade Science LO for Pressure/Buoyant force (Z=- 
0.28; df=51; p>0.05), Work and energy (Z=-0.79; df=19; p>0.05), and a ninth grade 
Geography LO, Map scales (Z—0.14; df=23; p>0.05). However, the groups' pre and post test 
differences differed in the study with the seventh grade Science LO for Electricity in daily life 
(Z—2.29; df=26; p<0.05); indicating that the individually studied students benefitted from the 
LO more than the collaborative students. 

In addition, the researchers’ observation of the students' interactions with the LOs and with 
each other, and the small usability questionnaires given to the students after they study with 
LOs provided imminent information shedding light on the LO use. It was observed that both 
groups of students in all four studies asked for help from the human mentor in the lab, rather 
than first referring to the help facilities of the LOs. Furthermore, the time of the experiments 
(30-70 minutes) was probably very short for the students to develop a personal or mutual 
strategic approach to the tasks given. The period was also too short to develop collaborative 
routines. The groups enjoyed the LOs, and they all finished the tasks of the LOs, nevertheless 
they wanted to study the LOs again, indicating that they had to spend more time with the LO 
environments. Though it is not a statistically significant difference, the collaboratively studied 
students seem to benefit less from the LOs possibly due to the fact that the immediate 
feedback either in the form of textual explanation or as the visual consequence of the 
simulations provided by the LOs moderated the effect of social interaction between the pairs. 
When a delayed form of feedback was preferred instead of the immediate form of feedback, 
the pairs would have elaborated their decisions more and engage in reflective negotiation 
more. 

Some of the students who worked in pairs advised the researcher to create groups based on 
students' peer preferences. Future research may extend the study time with larger number of 
tasks in the LOs, and let learners choose different partners at different stages of the studies. 
Moreover the observations of this study confirmed the recommendations that group size and 
group formation are critical. For effective collaboration, maximization of meaningful interaction 
may be enhanced by organizing pairs or groups heterogeneous in gender, culture, language 
background and ability (Tolmie et al, 2009). Since pairs in the collaborative groups had to be 
randomly assigned, the possibility of controlling heterogeneity of members was not possible. 
Future research should consider group size and heterogeneous group formations. It may be 
better to assign three students into groups rather than pairing two students; because it was 
observed in this study that one member of the dyadic pairs sometimes dominates the social 
and cognitive interaction causing the partner, as well as herself/himself, to engage less in 
epistemic activities such as describing, predicting, arguing and evaluating. 

Though there seems to be statistically insignificant improvement in the collaboratively studied 
groups' performances in the two LO studies (LO-2 and 4), the results of the statistical analysis 
of this study do, in general, agree with the studies in Ding (2009) and Edwards et al (1997). 
As Ding reported that one student was put at a disadvantage while the group (in this study 
pair group) succeeded. Also, as Edwards et al point out that some students work in a more 
consensual fashion than the others, debating their plans together before acting; some tend to 
act without discussing their ideas with their partners. Similarly, in this study, while most pairs 
in all four studies worked in a consensual manner, some pairs did not discuss both students’ 
ideas: the first idea produced by one of the partners was accepted which caused less 
cognitive engagement of another member of the pair. The researchers had to often notify 
those pairs to elaborate and to discuss mutually any opinions before trying out that opinion on 
the system. This ought to be considered in designing new collaborative version of the LOs, 
and the LOs for the pairs may have facilities (and requirements) for entering both students’ 
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ideas and sequentially test those presented ideas or entries. Further, this finding also 
confirms the assertion by Johnson & Johnson (1994) and Tolmie et al (2009) that initial 
training in group skills may serve to enhance benefits of collaborative learning. 

Although some of the pairs controlled the LO interface in turn, the pairs’ control of the LO 
interfaces created problems for some pairs: Both students wanted to control the interfaces, 
which caused them to waste time and dispute over control of the manipulative device, rather 
than argue about the content and the issue at hand. This problem was resolved promptly, 
however by enabling the use of multiple manipulative devices or having available Microsoft 
Surface like multiple-touch sensing interfaces which may prevent such problems. In fact, a 
recent study (Infante et al, 2009), employed a multiple-mouse controlled environment from 
which collaboratively studying learners benefitted more. 

It may be easy to suggest that teachers should intervene to regulate the collaboration 
between students, and the interaction between students and LOs. However, a teacher’s 
intervention in students’ work with LOs may impede the internal discourse a student is 
involved. Also, with the emergence and wide availability of LOs, the design of LOs should still 
mitigate teachers' preoccupation with concretizing the content for all students, but concentrate 
on the one in most need. Hence the LO should consider components which can engage 
students more, and keep records of students’ collaborations with others and interactions with 
the LO facilities with which students can (1) produce arguments and counter-arguments, (2) 
conduct experiments to support arguments, (3) and compare different arguments and 
opinions. The current work may have been compromised by the fact that the pre and post 
tests developed had not been pilot tested, and their reliability was not statistically ensured, but 
that only an expert opinion validated the tests. The statistical verification of properties of the 
tests to be used in future studies should be conducted. Hence the test results in this study 
should be interpreted accordingly. The extension of this work, which is currently being 
conducted, will fulfill the requirements noted in this study, and implement the emerging 
Common Cartridge standards: roles of each student of a pair for all activities will be scripted 
and written down on worksheets. Then the students will be asked and monitored to follow 
directions of the worksheets attached to LOs. It will employ revised and developed LOs under 
different study modes in order to examine their effect on long term retention of the learned 
material. 
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